Search for: All records

Vision-Language Models (VLMs) have made rapid progress in reasoning across visual and textual data. While VLMs perform well on vision tasks that they are trained on, our results highlight key challenges in abstract pattern recognition. We present GlyphPattern, a 954 item dataset that pairs 318 human-written descriptions of visual patterns from 40 writing systems with three visual presentation styles.GlyphPattern evaluates abstract pattern recognition in VLMs, requiring models to understand and judge natural language descriptions of visual patterns. GlyphPattern patterns are drawn from a large-scale cognitive science investigation of human writing systems; as a result, they are rich in spatial reference and compositionality. Our experiments show that GlyphPattern is challenging for state-of-the-art VLMs (GPT-4o achieves only 55% accuracy), with marginal gains from few-shot prompting. Our detailed analysis reveals errors at multiple levels, including visual processing, natural language understanding, and pattern generalization. 

Although LLMs are increasing the productivity of professional programmers, existing work shows that beginners struggle to prompt LLMs to solve text-to-code tasks (Nguyen et al., 2024; Prather et al., 2024; Mordechai et al., 2024). Why is this the case? This paper explores two competing hypotheses about the cause of student-LLM miscommunication: (1) students simply lack the technical vocabulary needed to write good prompts, and (2) students do not understand the extent of information that LLMs need to solve code generation tasks. We study (1) with a causal intervention experiment on technical vocabulary and (2) by analyzing graphs that abstract how students edit prompts and the different failures that they encounter. We find that substance beats style: a poor grasp of technical vocabulary is merely correlated with prompt failure; that the information content of prompts predicts success; that students get stuck making trivial edits; and more. Our findings have implications for the use of LLMs in programming education, and for efforts to make computing more accessible with LLMs. 

The dispersion of plant pathogens, such as rust spores, is responsible for more than 20% of global crop yield loss annually. However, the release mechanism of pathogens from flexible plant surfaces into the canopy is not well understood. In this study, we investigated the interplay between leaf elasticity and rainfall, revealing how a flexible leaf structure can generate a lateral flow stream, with embedded coherent structures that enhance transport. We first modeled the linear coupling between drop momentum, leaf vibration, and the stream flux from leaf surfaces. With Lagrangian diagnostics, we further mapped out the nested coherent structures around the fluttering profile, providing a dynamical description for local spore delivery. We hope the mechanistic details extracted here can facilitate the construction of physically informed analytical models for local crop disease management. 

Abstract Reconfigurable hybrid nanoparticles made by decorating flexible polymer shells on rigid inorganic nanoparticle cores can provide a unique means to build stimuli‐responsive functional materials. The polymer shell reconfiguration has been expected to depend on the local core shape details, but limited systematic investigations have been undertaken. Here, two literature methods are adapted to coat either thiol‐terminated polystyrene (PS) or polystyrene‐poly(acrylic acid) (PS‐b‐PAA) shells onto a series of anisotropic gold nanoparticles of shapes not studied previously, including octahedron, concave cube, and bipyramid. These core shapes are complex, rendering shell contours with nanoscale details (e.g., local surface curvature, shell thickness) that are imaged and analyzed quantitatively using the authors' customized analysis codes. It is found that the hybrid nanoparticles based on the chosen core shapes, when coated with the above two polymer shells, exhibit distinct shell segregations upon a variation in solvent polarity or temperature. It is demonstrated for the PS‐b‐PAA‐coated hybrid nanoparticles, the shell segregation is maintained even after a further decoration of the shell periphery with gold seeds; these seeds can potentially facilitate subsequent deposition of other nanostructures to enrich structural and functional diversity. These synthesis, imaging, and analysis methods for the hybrid nanoparticles of anisotropically shaped cores can potentially aid in their predictive design for materials reconfigurable from the bottom up.